Many applications and games in augmented reality devices and scenarios add augmented content, such as 2D overlays, text, virtual objects, etc. Augmented content may also be added to other types of applications, such as 2D overlays and text over an image. Some of these approaches use the location of the user and the orientation of the user's mobile device to add 2D overlays over the user's view. For location-specific apps, e.g., archaeological sites, museums, manufacturing floors, projection mapping, etc., the geometry of a site is sometimes computed in advance for use in positioning such overlays within an image or scene. Further, recent work in this area has also used 3D geometry recovered from a scene to position overlays and objects within the scene. In such cases, the recovered 3D geometry is often used to demonstrate various physical simulation examples or to provide various 2D or 3D graphical or textual overlays and the like for use in augmenting such scenes.